Generative models for discovering sparse distributed representations
نویسندگان
چکیده
منابع مشابه
Generative models for discovering sparse distributed representations.
We describe a hierarchical, generative model that can be viewed as a nonlinear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual inference correctly. Once perceptual inference has been performed the connection strengths can be updated using a very simple learning rule that only ...
متن کاملDiscovering Discrete Distributed Representations
Competitive learning is an unsupervised algorithm that classifies input patterns into mutually exclusive clusters. In a neural net framework, each cluster is represented by a processing unit that competes with others in a winnertake-all pool for an input pattern. I present a simple extension to the algorithm that allows it to construct discrete, distributed representations. Discrete representat...
متن کاملDiscovering Binary Codes for Documents by Learning Deep Generative Models
We describe a deep generative model in which the lowest layer represents the word-count vector of a document and the top layer represents a learned binary code for that document. The top two layers of the generative model form an undirected associative memory and the remaining layers form a belief net with directed, top-down connections. We present efficient learning and inference procedures fo...
متن کاملEnergy-Based Models for Sparse Overcomplete Representations
We present a new way of extending independent components analysis (ICA) to overcomplete representations. In contrast to the causal generative extensions of ICA which maintain marginal independence of sources, we define features as deterministic (linear) functions of the inputs. This assumption results in marginal dependencies among the features, but conditional independence of the features give...
متن کاملSparse Additive Generative Models of Text
Generative models of text typically associate a multinomial with every class label or topic. Even in simple models this requires the estimation of thousands of parameters; in multifaceted latent variable models, standard approaches require additional latent “switching” variables for every token, complicating inference. In this paper, we propose an alternative generative model for text. The cent...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences
سال: 1997
ISSN: 0962-8436,1471-2970
DOI: 10.1098/rstb.1997.0101